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ABSTRACT 

The Students Upgrading through Computer and Career 
Education Systems Services (Project SUCCESS) program in New York City 
was evaluated under Title VII of the Elementary and Secondary 
Education Act. Project SUCCESS targeted three high schools with a 
recent influx of immigrant students. The project's design included 
instruction in English as a Second Language (ESL) , native Language 
arts (NLA) in Chinese and Spanish, and bilingual instruction in 
content area subjects. The design also incorporated career 
orientation services, academic advising, and college consultation for 
target students, as well as summer enrichment programing. In 199^-92, 
the project enrolled 438 limited Englir:h proficient high school 
students. Project SUCCESS was fully implemented and met its 
objectives for Chinese NLA and career development, as well as for 
Spanish NLA, parental involvement, and staff awareness of student 
needs. Objectives for ESL, staff development, attitude change, and 
content area subjects were not fully met. Dropout prevention and 
attendance objectives were partially met. Recommendations are made 
for the following year to meet all program objectives. Appendix A 
describes data collection and analysis, and Appendij B describes 
instructional materials. (SLD) 
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FOREWORD 



The body of this report is preceded by an Extract which presents an overview of salient points of 
the project: funding cycle; enrollment figures; background of students served; admission criteria; and 
programming features, strengths, and limitations, including the outcome of all objectives. The extract also 
presents the conclusions drawn by the Office of Research, Evaluation, and Assessment (OREA) about the 
program and its recommendations for program improvement. 

The extract is followed by the body of the report, titled Program Assessment . This includes such 
information as staffing, program Implementation, and outcome and implementation objectives, 
Instructional objectives are presented first, followed by non instructional objectives. The report then 
addresses those aspects of programming mandated by Title VII regulations that do not have specifically 
stated objectives. This may be information on attendance and dropout rate, grade retention, 
main St reaming, referrals out of the program to meet special needs of the students, and withdrawals. A 
case history concludes the report. 

Data for this profile were collected and analyzed using a variety of procedures, which are 
described in Appendix A following the text. 
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EXTRACT 

PROJECT DIRECTOR: Mr. Tim Law FUNDING CYCLE: Year 2 of 3 

SITES 



High School Borough Enrollment* 

(fall) (spring) 

Murry Bergtraum Manhattan N/A 56 

Edward R. Murrow Brooklyn 174 155 

John Dewey Brooklyn 123 126 



*The project enrolled 438 students (63 more than in the previous year), serving 297 in the fall and 367 in the 
spring. Male students numbered 109, female 111; gender was not suted for 218. 



STUDENT BACKGROUND 



Native Language Number of Students Countries of Origin Number of Students 

Chinese 308 China 238 

Cantonese 271 Russia 71 

Mandarin 37 Hong Kong 45 

Russian 70 Haiti 8 

Spanish 27 Panama 7 

Haitian 5 Burma 5 

Hebrew 4 Greece 5 

French 3 United Stales 5 

Other 8 Vietnam 5 

Unreported 13 Other 33 

Unreported 16 



Median Years of Education in Native Country: 7.0; in the United States: 3.0 
Percentage of Students Eligible for Free Lunch Program: 48.8 



ADMISSION CRITERIA 

The project admitted students who scored at or below the 40th percentile on the Langr,age Assessment 
Battery (LAB). Other factors taken into account were home language, high school application:, remediation 
eligibility status, evaluation of foreign transcripts, performance on teacher-made tests for native language 
proficiency, performance on New York City Regents Competency Tests (R.C.T.s), and results of a guidance 
placement interview. Parental permission was required 



PROGRAMMING 



Design Features 

Project SUCCESS targeted three high schools vtith a recent influx of immigrant students. The project's 
design included instruction in English as a Second Language (E.S.L.), Native Language Arts (N.L.A.) in Chinese 
and Spanish, and bilingual instruction in content area subjects. The design also incorporated career orientation 
services, academic advisement, and college/university consultation for target students. The project proposed to 
inaease parents* awareness of career opportunities open to their children. The design also incorporated plans for 
summer enrichment programming. 

Capacity building. It is expected that tax-levy funds will be used as of September 1995 to fund a bilingual 
career specialist, bilingual teaching materials, and extracurricular activities. 



Strengths and Limitations 

Staff helped newly-arrived LEP students make a smooth transition into the New York City school system. 
Tbe project provided career orientation services, academic advisement, and college consultation to program 
students. However, it did not attempt to increase parents' awareness of career opportunities open to their 
children. 



CONCLUSIONS AND RECOMMENDATIONS 

Project SUCCESS was fully impl-^.mented. As in the previous year, it met its objective for Chinese 
N.L.A., its career development objectives for meeting with the career spiecialist and for holding a career 
conference, and its objective for field trips. It also met three objectives which had not been evaluated the 
previous year for lack of data: Spanish N.L.A., parental involvement, and staff awareness of pupil needs. The 
project failed to meet its objectives for E.S.L., staff development, attitude toward school and heritage, and content 
area subjects, all of which it had met in the previous year. However, the project came close to meeting its 
content area objective, failing only at John Dewey High School in the fall for mathematics and in the spring lor 
science. As in the previous year, the project partially met its attendance objective, meeting the objective ai .>nc 
high school, coming close to meeting it at a second, and failing to provide the data for the third site. It met its 
dropout objective at one high school (having failed to meet it at any site in the previous year). The dropout rate 
for project students was lower than mainstream students at all three sites. The project did not offer information 
on sununer programming. 

The conclusions, based on the findings of this evaluation, lead to the following recommendations 

• Explore E.S.L. techniques which have proven to be successful, and develop them in the classrcv>m and 
in staff workshops. 

• Provide after-school individualized instruction and increase peer tutoring at John Dewey High S^^htK)! 
to assist students in mathematics and science. 

• Increase efforts to impress upon students the importance of remaining in school and attending 
regularly. 

• Hold discussion groups with the intent of inaeasing students* positive attitudes about their heritage 
and schooling. 



PROGRAM ASSESSMENT 



STAFFING 



Title VII Suff (TolaM) 

Title 
Funding 

Project Director 
Bilingual Career Specialist 
Resource Specialist 
Resource Teacher 



Degree 



P.D. (Professional Diploma) 

M.A. 

MA. 

B.A. 



Lanpuai^e Competencies 



Chinese 
Chinese 
Chinese 
Chinese 



Title VII 



Full time 
Full time 
Full time 
Full lime 



Other Staff Working with Project Students (Total 42) 

Language Competencies and Teaching/ 
Title Degree Communicative Proficiencies (TP/CPV 

Teacher 37 Ph.D. 2 Spanish TP 10, CP 1 

Paraprofessional 5 M.A./M.S. 31 Chinese TP 7, CP 4 

B.A/B.S. 8 French TP 1 

A.A 1 luhan TP 1 

Project SUCCESS did not provide complete information on certiflcations or language competencies. 



IMPLEMENTATION AND OUTCOMES (Objectives prefaced by •) 
English as a Second Language LE.S.L,) 

E.S.L, CLASSES 

High School Level Periods Weekly Students Enrolled 



Edward R. Murrow 


Beginning 


5 


17 




Intermediate 


5 


68 




Advanced 


5 


43 




Transitional 


5 


42 


Murry Bergtraum 


Beginning 


15 


23 




Intermediate 


10 


39 




Advanced 


5 


59 




Transitional 


5 


9 


John Dewey 


Beginning 


15 


13 




Intermediate 


10 


32 




Advanced 


10 


27 




Transitional 


5 


23 



Teacners employed a participative, interactive style of learning (i.e. stoiy telling, role-playing, and games). 
See Appendix B for a list of instructional materials used. 



*Teaching Proficiency^ (TP): Competent to teach in this language. 
Communicative Proficiency (CP): Conversational capability only. 
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• Seventy percent of the target students will demonstrate an appropriate increase in English proficiency. 
Evaluation Instrument: Language Assessment Battery (LAB)* 



Pretest: May 1991; posttest: May 1992. 

Number of students for whom pre- and posttest data were reported: 312 

Percent of students with pretest/posttest gains: 6L2 

Mean gain: 5.3 N.CE.s (s.d, = 10,4) 

Mean gain is statistically significant (r = 9.06, p<.05). 

Project did not meet E.S.L. objective. Less than the proposed 70 percent of the students showed a gain 
in English language proficiency, although the mean gain was substantial and statistically signilicant. The project 
director felt that newly-arrived immigrants needed more time to learn and practice their new language. He stated 
that more E.S.L. materials would be purchased, peer-tutoring and afterschool programs would be established, a 
small-group E.S-L. tutoring program would be initiated, and conferences with E.S.L. teachers regarding students' 
progress would be held weekly. 

Native I^ngua^c Arts (N.L.A.'^ 

Students lacking native language literacy skills: 50 percent. 

Paraprofessionals were available to explain terminology when necessary. Cooperative learning was a 
motivating technique for encouraging participation and maintaining interest. 

N.L.A. CLASSES 

High School Level Periods Per Week Students Enrolled 

Spanish Chinese 

Munry Bergtraum Beginning 5 30 

Intermediate 5 - 30 

Advanced 5 - 45 

Advanced Placement 5 

Edward R. Murrow Beginning 5-4 5 

Intermediate 5 ' 39 35 . ' 

Advanced 5 28 20 

Advanced Placement 5 24 15 

John Dewey Beginning 5 - 22 

Intermediate 5 - 32 

Advanced 5 31 



•OREA used a gap reduction design to evaluate the effect of supplementary instruction on project students' 
performance on the LAB. Since all LEP students in New York City are entitled to such instruction, no valid 
comparison group exists among these students^ and OREA used instead the group on wh*ch the LAB was 
nonned. Test scores are reported in Normal Curve Equivalents (N.CE.s), which are normalized standard scores 
with a mean of 50 and a standard deviation of 21.06. It is assumed that the norm group has a zero gain in 
N.CE.s in the absence of supplementary instruction and that participating students* gains are attributable to 
project services. 
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• Seventy-five percent of the Chinese-dominant partidpants will demonstrate a significant increase in 
Chinese. 

• Seventy-five percent of the Spanish dominant participants will demonstrate a significant increase in 
Spanish language achievement. 

Evaluation Indicator: final course grades. 



PASSING GRADES IN NATIVE LANGUAGE ARTS 

Langua^re Fail S pring 

Enrollment Percent Passing Enrolhrient Percent Passing 

Chinese 203 92.6 207 93.7 

Spanish 12 91.7 10 100,0 

•p<.05 

Project met N.L^. objectives for both Chinese and Spanish. 



Content Area Subjects 

Instruction in content area subjects was taught with a biiingual methodology. Class notes and homework 
assignments were developed in Chinese and Spanish, stressing difOcult vocabulary. Visual aids such as videotapes, 
maps, and pictures further clarified difOcuh subject matter. Instructors encouraged students to work together and 
used peer tutors to encourage this. 

See Appendix B for a list of instructional materials. 



• At least 70 percent of all targeted students will score at or above the 65 percent passing criterion in 
the subject areas of social studies, mathematics, and science. 

Evaluation Indicator: final course grades. 

. PASSING GRADES IN CONTENT AREA COURSES 

Fall Spring 
High School Subject Enrollment Percent Passing Enrollment Percent Passing 



Murry Bcrgtraum 


Nfathematics 


46 


913 


62 


93.5 




Science 


19 


100.0 


25 


100.0 




Social Studies 


66 


97.0 


76 


100.0 


Edward R, Muriow 


Mathematics 


165 


84.8 


159 


88.7 




Sciecce 


162 


82.1 


158 


85.4 




Social Studies 


165 


84.2 


161 


87.0 


John Dewey 


Mathematics 


87 


67.8 


109 


69.7 


Science 


84 


72.6 


97 


68.0 




Social Studies 


88 


76.1 


115 


75.7 



Project partially met content «rea subject objective. The objective was met at all sites in all subject areas 
except at John Dewey High School in the fall for mathematics and in the spring for science. The project director 
stated that workshops in teaching methods would be held for the bilingual mathematics and science teachers of 
project students. 
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Career Developraent 



^ All graduating students will meet with the bilingual career specialist for advisement at least three 
times during the school year. 

Evaluation Indicator: program records. 

Students met with the bilinguil career specialist for career and college advisement on at least three 
occasions. Students with special needs were seen more frequently. 

Project met career dcvclopmcat objective for adviscmcot 



• 'Hie program will organize at least one conference for students in which the representatives of 
business and industry will present information on career options and skill requirements. 

Evaluation Indicator: program records. 

An Annual Career Day was held ai Long Island University's Brooklyn Campus. Guest speakers included 
lawyers, doctors, accountants, and architects. 

Project met career development objective for holding a career conference. 



Field Trips 

• The program will organize at least three field trips for the target students at the three sites, to 
historical museums. United Nations, White House, etc., to increase their familiarity with American 
culture and citizenship. 

Evaluation Indicator: program records. 

Project students visited the United Nations, Metropolitan Museum of Art, and the Pan-Asian Theater's 
production of "Fairy Bones." 

Project met field trip objective. 

Attendance 

# As a result of participating in the program, students' attendance will be significantly higher than 
mainstream students'. 

Evaluation Indicator: program records. 

ATTENDANCE RATCS 
High School Project Students Mainstre?im Students Difference 

Murry Bergtraum N/A 85.4 

Edward R. Murrow 91.2 89.7 1.5 

John Dewey 943 89.6 4.7 • 

•p<.05 

P ject met attendance objective at John Dewey High School only. The project student aUendancc rate 
was highe. ban the mainstream rate at Edward R. Murrow High School, but not significantly so. Tbe project did 
not supply attendance data for project students at Murry Bergtraum High School. 
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D ropout 



• Program students will have a significantly lower dropout rate than similar non-program students. 
Evaluation Indicator: program records. 

DROPOUT RATES 

High School Project Students Mainstream Students Difference 

Murry Bergtraum 0.0 3,5 3,5 * 

Edward R, Murrow 05 1.6 1,1 

John Dewey 0.7 1.8 l.l 

*/?<.05 

Project met dropout prevention objective at Murry Bergtraum High School only. While the dropout rate 
for project students was lower than mainstream students* at the other sites, it was not significantly so. However, 
mainstream dropout rates at these two sites were very low, and it might have been difGcult to achieve significantly 
lower rates. 



Attitude Toward Heritage 

• Seventy-five percent of ail target students will demonstrate an improvement in attitude toward their 
heriuge. 

Evaluation Instrument: OREA-developed cultural heritage attitude scale (5-point Likert scale), 

OREA received data on 357 students, 56,9 percent of whom indicated an improvement in attitude toward 
their heriuge. 

Project did not meet objective for improvement in attitude toward heritage. The project director stated 
that community members would hold workshops and meetings for students to increa5e appreciation of various 
heritages more fully. 

Attitude Toward School 

• Seventy-five percent of all target students will demonstrate an improvement in attitude toward school. 
Evaluation Instrument: OREA-developed scale of attitude toward school (5-point Likert scale), 

OREA received data for 357 students, 615 percent of whom indicated an improvement in attitude toward 

school. 

Project did not meet objective for improved attitude toward school. The project director stated his 
intention to arrange for professionals to speak at school assemblies in the hope of inspiring students to improve 
their attitude toward school. 



Grade Retention 

Twenty-one project students (4.8 percent) were retained in grade. No students had been reported as 
retained in the previous year. No objective was pro{X)sed in this area. 



Enrollment in Post-secondary Instituticas 

The project reported that 56 participating students enrolled in college after graduation. No objective was 
proposed. 

Students With Special Academic Needs 

Referral to special education. No students were referred to special education. 

Remedial programs. No students were referred to remedial programs. 

Gifted and talented programs. No students were referred to gifted and talented programs. 

Nc objective was proposed in any of these areas, and no referrals were made in the previous year. 

Mainstreaming 

Nine students (2.1 percent of participants) were mainsireamed. The previous year, one student (0,3 
percent) had been mainstreamed. No objective was proposed. 

Academic performance of former project students in mainstream. The project did not submit data on ibe 
academic performance of the student mainstreamed the previous year. 

Staff Development 

• Ninety percent of the program staff members will enroll in at least one university course each 
semester. 

Evaluation Indicator: Program records 

The project director reported that all Title Vll-funded staff enrolled in at least one university courw 
during the year under review but most took a course for one semester only. 

Project did not meet staff development objective. 

Staff Awareness 

• Eighty percent of the staff in the program will demonstrate an increase in awareness of pupii needs 
and problems. 

Evaluation Instrument: OREA-developed scale of increase in awareness of pupil needs and problems (5 ptMni 
Likert scale). 

OREA received data for four staff members, all of whom indicated that they had increased their 
awareness of pupil needs and problems. 

Project met objective for increased staff awareness of pupil nccdit and problems. 

Curriculum Development 

Project staff developed curricula in sequential niithematics, conceptual science, economics, and global 
history. (An objective for curriculum development was proposed for the third project year only,) 
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Parental Involvement 



Parent meetings were scheduled as necessary to discuss student progress and parent concerns. 

• The proportion of program students' parents who participate in Open School Day /Evening will be 
" equal to or greater than the proportion of mainstream students' parents who participate in this 
activity. 

Evaluation Indicator: attendance records. 

PERCENT OF PARENTAL ATTENDANCE 

High School Program Parents Mainstream Parents 

Murry Bergtraum 98 88 

John Dewey 96 90 

Edward R. Murrow 95 89 

Project met objective for parental involvement. 

CASE HISTORY 

M. came to the United States in 1988 and entered the 9th grade at John Dewey High School the 
following year, A LEP student, he was placed in the bilingual program. He often visited the bilingual resource 
center for counseling and tutoring, and his parents worked with the coordinator of the bilingual resource cenier. 
He became a member of the school mathematics team and went on to participate in nationwide mathematics 
contests. His LAB scores went from the 28th to the 60th percentile. 



APPENDIX A 



DATA COLLECTION AND ANALYSIS 



COLLECTION 

OREA evaluation consultants visit sites and interview key personnel. The project director gathers data 
and, with the consultant, completes forms (as shown below) as necessary. 

Student Data rorm 

This one-page form is filled out by staff for each participating and mainstreamed student. OREA gathers 
data from this form on backgrounds, demographics, academic outcomes, attendance, referrals, and exit from tbe 
program. 

Project Director's Questionnaire 

The Project Director's Questionnaire includes questions on staff qualifications, program implementation, 
periods of instruction, and instructional materials and techniques. 

Project Director's Interview 

The interview gathers information on program and student or staff characteristics not supplied by the 
Project Director's Questionnaire. The interview also allows project staff to offer qualitative data or amplify 
responses to the questionnaire. 

Citywide Test Scores 

OREA retrieves scores centrally from the Language Assessment Battery (LAB) and other citywide tests. 
For evaluation purposes, these test scores are reported in Normal Curve Equivalents (N.C.E.s). N.C.E.s are 
normalized standard scores with a mean of 50 and a standard deviation (s.d,) of 21.06. They constitute an equal- 
interval scale in which the distance is the same between any two adjacent scores. A gain of 5 N.CE.s is the same 
whether it is at the lower or the higher end of the scale. N.C.E.s can be used in arithmetic computations to 
indicate group progress. (Percentile scales, ahhough more familiar to many, are unsuitable for such computations 
since they are not equal-interval.) 

Likert'Tvpe Surveys 

Likert-type surveys, in which respondents mark their opinions on a scale from one to five, are used in a 
variety of ways. They examine student attitudes (i.e., toward school and career, native language use, and native 
and mainstream cuhures). They also assess staff and parent attitude and reactions to workshops and other 
activities. 



ANALYSIS 

Gap Reduction Evaluation Design 

OREA uses a gap reduction design for measuring changes in standardized tests. Since no appropriate 
non-project comparison group is available in New York City, where all students of limited English proficiency 
(LEP) are entitled to receive supplementary services, OREA compares the progress of participating students with 
that of the group on which the test was normed. It is assumed that the norm group would show a zero gain in 
the absence of instruction, and gains made by project students could be attributed to project services. Scores arc 
reported in Normal Curve Equivalents (N.C.E.S), which are normalized standard scores with a mean of 50 and a 
standard deviation of 2L06- (See "Citywide Test Scores" above.) 

To test whether pre/posttest gains are greater than could be expected by chance alone, OREA uses a /- 
test. To test whether a difference between two proportions (e.g., program and mainstream attendance rates) is 



greater thaa could be expected by chance, ORE A uses a z-test and reports the dfifereaces between ibe two 
proporticDS. The level of significaoce is set at .05 for all tests. 

Techniques For Mi "^nni7ing Krrnr 

The evaluation procedures minimize error by providing for proper administration of evaluation 
instruments through a combination of testing at 12-month intervals, appropriate analysis procedures 
and reporting. 

Instruments of measurement include the LAB (see above), the Degrees of Reading Power (D.R.P.) test, 
the Metropolitan Achievement Test- Mathematics (MAT-Math), El Examen de Lectura en Espanol (ELE), 
Likert-type scales (see above), and project-developed tests. Except for Likert scales and project-developed tests, 
these instruments are scored on a citywide basis at the Scan Center of the New York City Public Schools. 



APPENDIX B 



INSTRUCTIONAL MATERIALS 



Science 



Biology and Human Process 
RCT Science Rex^iew for NYC 



Charles Tanzer 



Prentice Hall 
Prentice Hall 



Mathematics 



Integrated Mathematics 
Jos Mathematics 



Dressier &. Keenan 



Amsco 

Addison Wesly 



Social Studies 

Eastern Hemisphere: Europe, Asia, Africa, Oceana Schreiber, Steper, Patrich Foresman, Scott 
El mundo y su gente Cooper, Kenneth Silver Burdetc Co, 



Chinese 



Chinese Text 
Chinese Text 



Chinese Culture Press 
Government of Hong Kong 



E.S.L, 



Achieving Competence in Reading & Writing H.L. Christ 

Impact'50 Short Stories F, Safier 

Outsiders-American Short Stories J.S. Mullen 



Amsco 

Harcouri Brace Jovanovich 
Prentice Hall Regents 



Typing 



Series 7 Typing 
Century 21 Typing 



Gregg-McGraw Hill 
Southwestern 



